The present invention relates to processing elements in neural networks. In particular, the present invention provides for digital neural processing elements that simulate neuron electrophysiology at subcellular levels. These digital neural processing elements when connected according to a neural topology may be used to solve a wide range of signal processing and data processing problems.
Artificial neural networks are computational mechanisms that are structured and act in a manner that is analogous to biological neurons. In its simplest form, an artificial neural network consists of a number of Neural Processing Elements (NPEs) interconnected via weighted connections. In artificial neural networks, the processing elements are typically called “neurons” and the connections are called “synapses”. The signal lines from the neurons to synapses are called “axons”, and those from the synapses to the neurons are called “dendrites”. These elements are the “Standard Components” of the neural system. Each neuron is impinged upon by numerous synapses, which carry signals from neighboring neurons. The input signals are integrated by the dendrites of the neuron in the form of what are called the “post synaptic potential” (PSPs) until a critical threshold is reached, at which point an “action potential” (AP) is generated. In biological systems, the PSP is a continuous in time, continuous in amplitude signal, and the AP is a continuous in time, discrete in amplitude signal. The action potential is propagated down the axon to a synapse connecting the neuron with another neuron.
Although these elements can be quite simple, the large amount of them required to perform practical tasks makes their implementation complicated. The most common PE, in which incoming signals are summed and then convolved with a sigmoidal transfer function, embodies two basic assumptions. First, it is assumed that communication between neurons can be approximated by a slowly-varying scalar value, which is typically interpreted as the frequency of APs. Second, it is assumed that computation within neurons occurs instantaneously—i.e., that there is no delay between the arrival of an incoming signal and its effect on the output. While these assumptions are biologically unrealistic, they are generally viewed as providing an adequate approximation that is relatively straightforward to implement, from which practical computational studies can be effected.
Thus, classic neural network models employ basic computational units with no internal structure and a fixed temporal window for signal processing. Cable theory, in contrast, predicts that biological neurons integrate inputs differentially depending on the synaptic position on the dendritic tree. Specifically, the farther a connection is from the soma, the longer are the signal onset and duration, and the lower is the peak value. These electrotonic effects on dendritic integration are due to the interplay of passive membrane properties (resistance and capacitance). While many neuron classes in the central nervous system possess a variety of active (voltage-gated) channels on their dendrites, there is considerable experimental support for the basic passive mechanisms underlying dendritic integration. The connectivity of most major natural networks in the mammalian brain is organized in layers, such that specific feedforward, recurrent, and feedback projections each contact the target dendrites within precise ranges of electrotonic distances. This layered organization is observed, for example, in all major pathways in the cerebral cortex, hippocampus, and cerebellum. It is thus likely that dendritic integration and the anatomical organization of synaptic connectivity together constrain network dynamic, and, ultimately, plasticity and function.
Detailed dendritic passive models of single neurons are widely adopted in computational neuroscience, often complemented with active mechanisms. However, for larger scale simulations, simplified integrate-and-fire models are frequently adopted instead. Several types of integrate-and-fire models allow the inclusion of electrotonic effects on dendritic processing (e.g. in the form of time constants for signal rise and decay). However, large scale models are not built with layered synaptic architectures (e.g., with principal cells receiving proximal feedforward excitation, distal recurrent collaterals, and intermediate lateral inhibition) and corresponding electrotonic parameters. The activity dynamics in biological networks are exquisitly constrained by the connectivity and its relationship to electrotonic architecture. This relationship is presently not captured in available artificial neural network models.
What is needed is a neural processing element capable of modeling biological neurons with respect to electrotonic effects on dendritic integration. This neural process should be capable of simulating these electrotonic effects on the spike response dynamics of networks including feedforward and recurrent excitation.